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Abstract. Let G be a classical complex Lie group, P any parabolic subgroup 
of G, and G/P the corresponding partial flag variety. We prove an explicit 
nonrecursive Giambelli formula which expresses an arbitrary Schubert class in 
H*(G/P) as a polynomial in certain special Schubert class generators; in the 
even orthogonal case this depends on the main result of jBKT4| . Our formula 
extends to one that applies to the torus-equivariant cohomology ring of G/P 
and to the setting of symplectic and orthogonal degeneracy loci. 



0. Introduction 

The classical Giambelli formula [G] is one of the fundamental results concern- 
ing Schubert calculus in the cohomology ring of the usual Grassmannian X. The 
cohomology of X has an additive basis of Schubert classes, and is generated as a 
ring by certain special Schubert classes, which are the Chern classes of the universal 
quotient bundle over X. The formula of Giambelli writes a general Schubert class 
as a determinant of a Jacobi-Trudi matrix with entries given by special classes. 

The Giambelli problem can be formulated for any homogeneous space G/P, 
where G is a classical complex Lie group and P a parabolic subgroup of G. One 
can choose as Schubert class generators for the cohomology ring H*(G/P) the 
puUbacks of the special Schubert classes on Grassmannians; typically these will 
be the Chern classes of the tautological quotient bundles over G/P. The question 
then is to find an explicit combinatorial formula which expresses a general Schubert 
class in H*(G/P) as a polynomial in the special classes. We proceed to give a brief 
survey of the work to date on the Giambelli problem for such homogeneous spaces. 

The case of the full flag variety G/P, where the cohomology is generated by 
Schubert divisors, is more amenable to study. Note that in general, the Giambelli 
polynomials are only defined up to the ideal of relations generated by the invariant 
polynomials under the action of the Weyl group of G. Bernstein- Gelfand-Gelfand 
|BGG| and Demazure [Dl[ ID2j used divided difference operators to construct an 
algorithm that produces polynomials which represent the Schubert classes. For 
the general linear group, Lascoux and Schiitzenberger [LSlj applied this method to 
define Schubert polynomials , a particularly nice choice of representatives with rich 
combinatorial properties. An explicit combinatorial formula for the coefficients 
of Schubert polynomials was given by Billey, Jockusch, and Stanley |BJS| - thus 
resolving the Giambelli question in the case of the complete flag variety in type A. 
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This was generalized to arbitrary GL„ partial flag varieties by Buch, Kresch, Yong, 
and the author [BKTYI §5]. 

In the other classical Lie types, one considers the varieties which parametrize 
flags of subspaces of a vector space which are isotropic with respect to an orthog- 
onal or symplectic form. Pragacz [P] found an explicit Giambelli formula for the 
maximal isotropic Grassmannians, in the form of a Schur Pfaffian [Schj . At the 
other extreme, for the full flag varieties G/B in types B, C, and D, the analogous 
theory of Schubert polynomials is not uniquely determined (see [BHl IFu21 IFK21 
ILPH ILP2[ ITli IT3| for examples) and their combinatorics is more challenging. 

Giambelli formulas for non-maximal isotropic Grassmannians were discovered 
only recently [BKT21 rBKT4j . Unlike most previously known examples, the nature 
of the formulas in loc. cit. is not determinantal - instead, they are expressed using 
Young's raising operators [Y] . In the present work we will generalize these results 
to settle the Giambelli problem for any classical G/P space (when G is an even 
orthogonal group, the problem is reduced to the main result of [BKT4]). One of our 
motivations for this is the fact that all the known Giambelli formulas expressed in 
terms of special Schubert classes have straightforward - often identical - extensions 
to the small quantum cohomology ring of G/P\ see [Hi EEl IFGP] lKT2| [KT3l 
IBKT3| . We expect that our new Giambelli formulas will similarly prove to be the 
right ones to look at in the quantum world. 

The modern formulation of the Giambelli problem is in the setting of an algebraic 
family of varying partial flag varieties, with applications to degeneracy loci of vector 
bundles. This story also has a long history, from the work of Thom-Porteous and 
Kempf-Laksov [KLj to the generalizations by Fulton, Pragacz, and others [Full 
lFiIllT\i3l[Fulim[lJTl[ICTl , IMNJ; see [EH [Mil E] for expositions. It has been 
known at least since the paper of Graham [Graj that the degeneracy locus problem 
for the classical groups from [Fu2|, IFu3' is equivalent to the Giambelli problem for 
the Schubert classes in the T-equivariant cohomology of G/P, where T denotes the 
maximal torus in G. In type A, the project culminated with the combinatorial 
understanding of the polynomials representing quiver loci jBFl IBKTYl IKMS] . 

Fulton asked in ,Fu3j for degeneracy locus formulas which have a similar shape 
for all the classical groups; moreover, that the formulas should be determinantal 
whenever possible. We give a complete answer to this question here, except that in 
addition to determinants we also require the raising operator Giambelli polynomials 
from [BKT2[|BKT4| . Furthermore, the results provide a combinatorial link between 
the general G/P Giambelli problem and the quiver formulas of [BKTY] . 

We now state one of our main theorems, referring to SJU [JSj and |j5]for the precise 
definitions. Equip the vector space E = C^" with a nondegenerate skew-symmetric 
bilinear form. Fix a sequence t) : di < ■ ■ ■ < dp oi positive integers with dp < n. 
Let X(j}) be the variety parametrizing partial flags of subspaces 

(1) E. : OcEiC--- CEpCE 

with dim Ei = di for each i and Ep isotropic. The Schubert varieties Xw in X(0) 
and their cohomology classes [Xw] are indexed by signed permutations w in the 
Weyl group of Sp2„ whose descent positions are included among the di. Let E'^ = 
E/Ep+i-j for I < j < p; the Chern classes of the E'^ are the special Schubert 
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classes in H*(X(c)), Z). We then have 

(2) [X„] QM)s^.{E'^ -E[)--- s^AE'p - E;_,) 

summed over all sequences of partitions A = (A^, . . . , A^) with A^ fc-strict, where 
k = n — dp. Here Q\i and s^i denote theta and Schur polynomials, respectively, 
and the coefRcient is a nonnegative integer which counts the number of p-tuples 
of leaves of shape A in the groves of the transition forest associated to O and w. 

Let IG(n — k, 2n) denote the Grassmannian parametrizing isotropic linear sub- 
spaces of E of dimension n — k. The morphism which sends E, to Ep realizes 
X{d) as a fiber bundle over IG(n — k, 2n) with fiber equal to a type A partial flag 
variety. The mixed nature of the ingredients in formula ^ is in harmony with 
this fact; however, this simple geometric observation cannot by itself produce an 
explicit solution to the Giambelli problem. 

Our proof of ([2]) and the corresponding formulas for degeneracy loci is mainly 
combinatorial. We work with the Schubert polynomials of Billey and Haiman [BHj . 
and more generally with their double versions introduced by Ikeda, Mihalcea, and 
Naruse |IMN| . These objects have many of the properties of the type A Schubert 
polynomials, but their connection with the geometry is less clear. This problem 
was addressed for the single Schubert polynomials in |T2[ IT3] and in [IMN] for 
their double counterparts. Using this theory, our geometric results follow readily by 
combining the Giambelli formulas for isotropic Grassmannians from [BKT2|,lBKT4| 
with new splitting theorems for these Schubert polynomials, which are analogues of 
[BKTYl Thm. 4 and Cor. 3] for the other classical Lie types. 

A first step towards splitting the Billey-Haiman Schubert polynomials was for- 
mulated by Yong [Yo| . however, his result does not suffice for the aforementioned 
applications to geometry. The point is that the flag of subspaces E, in ([T|) need not 
contain a Lagrangian (i.e., a maximal isotropic) subspace. In terms of the Weyl 
group, this is simply the fact that the flrst descent position of a signed permutation 
w need not be at zero. 

To solve this problem, we introduce the mixed Stanley functions, denoted by 
JwiX,Y) in type C. The functions Jw are defined as sums of products of type C 
and type A Stanley symmetric functions in two distinct sets of variables. For each 
fixed m > 0, they include among their coefficients the number of reduced words 
of a signed permutation w such that the last m letters in the word are positive 
(the letter is used to denote the sign change). When w has no descents smaller 
than k, where k > m, the - suitably restricted - are nonnegative integer linear 
combinations of theta polynomials indexed by fc-strict partitions A. It is the mixed 
Stanley coefficients that appear in this expansion which enter into the splitting and 
degeneracy locus formulas. Finally, our combinatorial interpretation of the num- 
bers in ^ depends on the transition equations of Lascoux and Schiitzenberger 
jLS3| and Billey B]. It would be interesting to have tableau formulas for the e^J', 
analogous to the ones in [BFl IBKTYi IKMS] for quiver coefficients. 

This article is organized so that most of the exposition is in type C; a final 
section explains the analogous picture in types B and D. We review the Schur, 
theta, and Schubert polynomials, as well as the Stanley symmetric functions we 
require in SJT] The mixed Stanley functions and their basic properties are studied in 
^ this includes applications to enumerating reduced words and product rules for 
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theta polynomials. Our splitting theorems for Schubert polynomials are proved in 
^31 and the applications to symplectic degeneracy loci and Giambelli formulas are 
deduced in ^ and 311 respectively. We conclude with the Schubert splitting results 
for the orthogonal groups in Sj6l 

I grateful to my collaborators Anders Buch, Andrew Kresch, and Alexander 
Yong for their hard work on related papers, especially [BKTY] and [BKT HlBKT2j . 
I also thank Leonardo Mihalcea for conversations about his joint paper |IMN| on 
double Schubert polynomials, which played an important role in this project. 

1. Preliminaries 

1.1. Schur and theta polynomials. An integer sequence is a sequence of integers 
a — (ai, Q!2, . . .) only finitely many of which are non-zero. The largest integer 
£ > such that 7^ is called the length of a, denoted i{a); we will identify an 
integer sequence of length i with the vector consisting of its first £ terms, and set 
|q;| = ^ ai. An integer sequence A is a partition if Xi > A^+i > for all i. We will 
represent partitions A by their Young diagram of boxes, let A' denote the conjugate 
(or transpose) of A, and write C A for the containment relation between two 
Young diagrams. Following Young jY], given any integer sequence a and natural 
numbers z < j, we define 

Rij{a) ^ {ai, . . . ,aj + 1, . . . ,aj - 1, . . .). 

A raising operator R is any monomial in these Rtj^s. If (ui,U2, ■ ■ ■) is any ordered 
set of commuting independent variables and a is an integer sequence, we let Ua = 
Yii '^ai , with the understanding that uq — 1 and u,. = if r < 0. For any raising 
operator i?, let Rua — Wija- 

Let c = (ci, C2, . . .) and d ~ (di, d2, ■ ■ ■) be two families of commuting variables. 
Define elements hr for r e Z by the identity of formal power series 

H-oo / oc \ / 00 \ 

^ hrt^=iY,i^iyc,f] ^(-w . 

r=-oo \i=0 / \i=0 ) 

Consider the raising operator expression 

i?°=n(i-i?.,) 

and for any partition A, define the Schur polynomial sx{c — d) by 

sa(c - d) = _R"/ia = det(ft,A.+j-i)ij. 

Let Y = (t/i, 2/2, ■ • ■) and Z = (zi, Z2, . . .) be two infinite sets of variables, and define 
the elementary symmetric functions er{Y) by the generating function 

00 00 

The super symmetric Schur function s\{Y/Z) is obtained from s\{c — d) by setting 
Cr — GriY) and dj. = er{Z) for all r > 1. The usual Schur 5- functions satisfy the 
identities sx{Y) = sx{Y/Z)\z=o and sx{0/Z) = sx{Y/Z)\y=o = (-I)I^Isa'(^). In 
particular, for each integer r, the function Sr{Y) is the complete symmetric function 
in the variables Y, also denoted hr{Y). 
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Fix an integer A: > 0. A partition A is k-strict if all its parts Xi greater than k are 
distinct; A is called strict if it is 0-strict. Any such A determines a raising operator 
expression i?^ by the prescription 



where the second product is over all pairs i < j such that A^ + Aj > 2k + j — i. 
Define elements gr for r G Z by the identity 



+ 00 



r— — oo \i—0 / \i—0 / 

and the theta polynomial 0a (c — d) by 

(3) ex{c-d)=R^gx. 

If A" = (.Ti,X2,...) is another infinite set of variables, the formal power series 
dr {X ; y ) for r G Z are defined by the equation 

CO ^ k oo 

i=l ' j = l r=0 

Following [BKT2j . we then set (dx{X ;Y) = R^-dx. The Qx for A fc-strict form a 
Z-basis for the ring F^^) = Z[i?i, i?2, • ■ •]• The ring F = F(°) is the ring of Schur Q- 
functions (see [M2[ III.8] and [Sch| ). and in this case ^?r(A) and 6a (A) are denoted 
by qr{X) and Qx{X), respectively. 



1.2. The hyperoctahedral group. Let Wn denote the hyperoctahedral group 
of signed permutations on the set {1, . . . ,n}. We will adopt the notation where 
a bar is written over an entry with a negative sign. The group Wn is the Weyl 
group for the root system B„ or C„, and is generated by the simple transpositions 
Si = + 1) for 1 < i < n — I and the sign change so(l) = 1. There is a natural 
embedding Wn ^ Wn+i defined by adjoining the fixed point n+l. The symmetric 
group Sn is the subgroup of Wn generated by the Si for 1 < i < n — 1, and is the 
Weyl group for the root system A„_i. We let 5*00 = U„S'„ and VFoo = U„W„. 

A reduced word for w G Wn is a sequence oi • ■ • of elements in {0, 1, . . . , n — 1} 
such that w = Sai ■ ■ ■ Sa^ and r is minimal (so equal to the length i{w) of w). Given 
any ui, . . . , Up, w G Woo, we write ui ■ ■ ■ Up = w if £(iti) + ■ • ■+£{up) = £{w) and the 
product of ui, . . . , Up is equal to w. In this case we say that ui • • • Up is a reduced 
factorization of w. We say that w has descent at position r > if > Wr+i, 
where by definition wq — 0. An element w G Woo is compatible with the sequence 
a : ai < • ■ • < flp of nonnegative integers if all descent positions of w are contained 
in a. A reduced factorization ui • ■ • Up = w is compatible with a if Uj{i) = i for all 
j > 1 and i < Oj-i. 

We say that a signed permutation w G Woo is increasing up to k if it has no 
descents less than k. This condition is vacuous ii k — 0, and for positive k it 
means that < wi < W2 < ■ ■ ■ < Wk- An important special case are the k- 
Grassmannian elements, which by definition satisfy i{wsi) = i{w) + 1 for all i ^ k. 
There is a natural bijection between /c-Grassmannian elements of Woo and fc-strict 
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partitions, obtained as follows. If w G Wn is /c-Grassmannian, there exist unique 
strict partitions u, v of lengths fc, r, and n — k — r, respectively, so that 

W = (Ufe, . . . ,Ul,Cl, ■ ■ • ,Cr>^^n-fc-r, ■ ■ • ,Vl)- 

Define /i^ for 1 < i < fc by 

fii^Ui + i- k- l + #{j I C] > Ui}. 

Then w corresponds to the fc-strict partition A such that the lengths of the first k 
columns of A are given by /ii , . . . , /i^, and the part of A in columns fc + 1 and higher 
is given by Conversely, for any fc-strict A the corresponding fc-Grassmannian 
element is denoted by w\. 

1.3. Schubert polynomials and Stanley symmetric functions. Following 
[FS| and [FKlj iFK2| , we will use the nilCoxeter algebra W„ of the hyperoctahedral 
group Wn to define Schubert polynomials and Stanley symmetric functions in types 
A and C, respectively. W„ is the free associative algebra with unity generated by 
the elements wq, mi, . . . , Un-i modulo the relations 

= i > ; 

UiUj — UjUi |i — il > 2 ; 

UiUi+iUt — Ui+iUiUi+i i > ; 

MOMlMoWl = UiUqUiUq. 

For any w G Wn, choose a reduced word ai • • ■ for w and define = . . . Ua/,- 
Since the last three relations listed are the Goxeter relations for Wn, it is clear 
that is well defined, and that the Uw for w S Wn form a free Z-basis of VV„. 
We denote the coefficient of Uw € Wn in the expansion of the element / G Wn by 
(/, w); thus / = Y.wew„ (/' ^) / e >V„. 

Let t be an indeterminate and define 

Ai{t) = (1 + tUn-l){l + tUn-2) ' ' ' (1 + tUi) ; 

Ai{t) = (1 - tui){l - tui+i) • • • (1 - tun-i) ; 

C{t) = (1 + tUn-l) • • • (1 + tUi){l + 2iUo)(l + tUi) • • • (1 + tUn-l). 

According to |FS[ Lemma 2.1] and [FK21 Prop. 4.2], for all commuting variables 
s, t and indices i, the relations Ai{s)Ai{t) = Ai{t)Ai{s) and C{s)C{t) — C{t)C{s) 
hold. If C{X) = C{xi)C{x2) ■■■ and A{Y) = Ai{yi)Ai{y2) • • • , we deduce that the 
functions and G^{Y) defined for w G Wn and zu G Sn by 

Fy,iX) = {CiX),w} and G^{Y) = {A{Y),vj) 

are symmetric functions in X and Y, respectively. The and F^, are the type A 
and type C Stanley symmetric functions, introduced in |Sta| and [BH[ IFK21 IL2| . 
We have that = Fj^-i. 

When is expanded in the basis of Schur functions, one obtains a formula 

(4) G^iY)= ^A5A(r) 

A : |A|=£(ro) 

for some nonnegative integers (see |LS2j . |EGj ). According to [LS2| (see also 
[Ml[ (7.22)]), we have = ■ Lascoux and Schiitzenberger |LS3j gave one 
of the first combinatorial interpretations for the coefficients c^, as the number of 
leaves of shape A in the transition tree T{w) they associated to uj. Equation ^ 
may be used to define the double Stanley symmetric functions Grxj(Y/Z). 
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For any tu g S'„, the Schubert polynomial of Lascoux and Schiitzenberger is 
given by 

(5) e^iY) = (Ai(yi)A2(y2)---A„_i(y„_i),tn). 

The definition ([5]) is equivalent to the one in [LSI] , as is shown in [FS| . Now define 

(6) €UX;Y,Z) - (i„_i(z„_i)---ii(zi)C(X)Ai(yi)---A„_i(2/„_i),ii;). 
If <^w{X ; Y) := £u)(X ;Y,0), then ([6]) is equivalent to the equation 

(7) G:„(X;r,Z)= J2 6u-^{-Z)€.{X;Y) 

uv—w 

summed over all reduced factorizations uv — w with u € Sn- The polynomials 
£iu in ^ were introduced by Ikeda, Mihalcea, and Naruse [IMN] : they are double 
versions of the type C Billey-Haiman Schubert polynomials [BHj . Their definition 
differs from but the equivalence of the two follows by combining ([7]) with |FK2[ 
§7] and [IMNi Cor. 8.10]. One checks that &^ and £^ are stable under the natural 
inclusion of Wn in Wn+i, and hence well defined for zu e 5*00 and w G Wo^, 
respectively. The &^{Y) for w e 5*00 form a Z-basis of the polynomial ring 
and the €w{X ; Y) for w £ Woo form a Z-basis of T[Y]. 

If n7 G 5*00 is a Grassmannian permutation with a unique descent at r, then 
the &^{Y) is a Schur polynomial in {yi, . . . ,yr). In |BKT2| §6], we obtained 
the analogue of this result for the (^^{X ;Y): if it; G Woo is fc-Grassmannian, 
then £i„(X;F) is a theta polynomial Q\{X -jY), where A is the /c-strict partition 
associated to w. 



1.4. Splitting type A Schubert polynomials. If r < s are any two integers, 
and P{X, Y, Z) is any polynomial or formal power series in the variables Xi, yj, 
and Zj, we let P[r, s] denote the power series obtained from P{X, Y, Z) by setting 
Xi = Q for all i if ^ [r, s] , yj = if j ^ [r, s] , and Zj = if — j ^ [r, s] . If G [r, s] 
we set = P[r, s] and P'") = P[0, s]. 

If w G Wn and v G 5*^, we define w x u G Wm+n to be the signed permuta- 
tion {wi, . . . , ui„, wi -|- n, . . . , -f n). For w G 5oo and 1 < r < s, equation ^ 
immediately gives 



(8) &^[r,s] 



6v{yr, ■ ■ ■ ,ys) if = Ir-l X w, 

otherwise. 



Given any sequence a : ai < . . . < Op of natural numbers, we furthermore obtain 

[1, ai]S„2 [fli + 1, • • • 6up [flp-i + 1, Op] 



til ■■■tip— 



summed over all reduced factorizations ui ■ ■ ■ Up = w. 

If zu is increasing up to r, then 6^ is symmetric in yi, . . . , j/^ and we have 

(9) 6W=gW= E crsA(2/i,...,y.) 

A:|A1=£M 



8 



HARRY TAMVAKIS 



with the coefficients as in (jl]). Suppose now that w is compatible with the 
sequence a, and set Yi — {yat^i+i, ■ ■ ■ ,yai} for each i. From the previous consider- 
ations, we deduce that 6^ satisfies the formula 

(10) 6^(r)= GuAYi)---Gu,{Yp) 

Ui ■■■Up—ZU 

summed over all reduced factorizations ui ■ ■ ■ Up = m compatible with a. Using (j4]) 
to refine (jlOp further gives 

(11) 6^(y)-^crsAi(yi)---.SA.(rp) 

A 

summed over all sequences of partitions A — (A^, . . . , A^), where 

- \^ r"i . . . r"" 
^\ — 2^ f-AP' 

U\-- -Up —zu 

summed over all reduced factorizations ui ■ ■ ■ Up = tu compatible with a. More 
general versions of pO|) . (fTTj) for universal Schubert polynomials [Fu4j and quiver 
polynomials [BF] are estabhshed in |BKTY|lKMS] . 

Following [BKTYi Cor. 3], a result of Fomin and Greene |FG] for the coefficients 
in Q implies an alternative formula for the . The column word col(r) of a 
semistandard Young tableaux T is obtained by reading the entries in the columns 
of T from bottom to top and left to right. For any zu G S^o compatible with a, the 
coefficient equals the number of sequences of semistandard tableaux (Ti, . . . , Tp) 
such that col(ri) • • ■col(Tp) is a reduced word for tn, the entries of Ti are strictly 
greater than Ui-i and the shape of Ti is conjugate to A* for each i. 

2. Transition for mixed Stanley functions 
2.1. Mixed Stanley functions. 

Definition 1. Given w £ Woo, the (right) mixed Stanley function J^(X;y) is 
defined by the equation 

MX;Y) = {C{X)A{Y),w) ^ ^ F„(X)G.(y) 

UV — W 

summed over all reduced factorizations uv = w with v G Soo- 

Definition [1] can be easily restated in terms of reduced decompositions and ad- 
missible sequences, along the lines of jBH| Eq. (3.2)]. One has a dual notion of a 
left mixed Stanley function J'^{X;Y) ~ J2uv=wGu-^{Y)Fy{X), summed over all 
reduced factorizations uv = w with u G 5*00 • This is equivalent to the right version 
since clearly J^{X;Y) = Jy^-i{X,Y). 

Recall that Jw^ = J^[0, /c]. If G Woo is increasing up to fc, we claim that 
cif'' = Jw"^ ■ Indeed, observe that if w = uv is a reduced factorization, then 
£{vsi) = i{v) + 1 for all i < k, i.e., v is also increasing up to k. It follows that 

(12) J2 Fu{X)&^^= F^{X)Gi'^^ = Ji'\ 

UV — W, vGSod UV — W, V^Soo 

as claimed. In particular, if w ~ wx is fc-Grassmannian, then 

(13) j«(X;y) = eA(X;y). 



A GIAMBELLI FORMULA FOR ISOTROPIC PARTIAL FLAG VARIETIES 



9 



Example 1. If A is a fc-strict partition, then 

(14) J^,(X;y) = ^i^;,/^(X)v(n 

summed over all fc-strict partitions /i C A. The function Fx/^^X) in (fT4| is the 
skew _F-function from 'T2' §5], which, when non-zero, is equal to F^^^-i{X). 

2.2. Transition equations. For positive integers i < j we define reflections tij € 
Soc and tij , tii G Woe by their right actions 

{...,Wi,...,Wj,.. .)Uj ^ {...,Wj,...,Wi,.. .), 
{...,Wi,...,Wj,...)tij = (. . . , Wj, . . . ,Wi, . . .), and 

(. . . , Wj, . . .)tu = (. . . , Wi, . . .). 

We let tji = tij. According to [B] Thms. 4, 5], the type C Schubert polynomials 
£u, = <Lw{X \ Y) satisfy the recursion formula 

(15) iw^yXwt,.,+ ^wu-A..-^ ^wuru.^ 

l<i<T i>l 

where r is the last positive descent of w and s is maximal such that Ws < Wr- If 
the last descent r of w satisfies r > fc, we deduce from ([72]) and (|15p that 

l<i<r i>l 

For any w 6 Woo which is increasing up to fc, we construct a rooted tree T^(w) 
with nodes given by elements of Woo and root w as follows. Let r be the last 
descent oiw. If w = 1 or r = fc, then set T^{w) — {w}. Otherwise, let s = max(i > 
r \ Wi < Wr) and ^{w) — <f>i(iy) U ^2{w), where 

$i(w) = {wtrstir I 1 < i < r and £{wtrstir) ~ ^(w)}, 
'&2(w) = {wtrstir I i > 1 and £{'wtrstir) — ^(w)}- 

To recursively define T''{w), we join w by an edge to each v G *I'(if), and attach to 
each V £ ^(w) its tree T^{v). We call T^{w) the k-transition tree of w. 

Lemma 1. T/ie k-transition tree T^{'w) is finite. All the nodes of T^{w) are 
increasing up to fc, and the leaves ofT*'{w) are k-Grassmannian elements. 

Proof. If w is fc-Grassmannian then clearly T^{'w) — w. We will show that if w 
is increasing up to k and not fc-Grassmannian, then $(w) ^ and all elements of 
$(w) are increasing up to fc. Let r > fc be the last descent of if, s be maximal such 
that Ws < Wr, and v = wtrs. If <&i(w) = 0, then according to Monk's criterion 
[Moj (see also [B] Lemma 1]), we must have Wi > Ws for all i < r. If > then 
virr G ^2{w). If Ws < 0, let « > be minimal such that Wi > —Ws- Then one 
can easily check using [B] Lemma 2] that vtir £ $2(w)- This proves that ^{w) is 
nonempty. 

To show that all elements of $(w) are increasing up to fc, we may assume that 
fc > 0. If there exists an i < fc such that £{vtir) = £{w), then Ws > Wi and for all 
j e (i,?"), Wj ^ {wi,Ws). It follows that vtir is increasing up to fc. If £{vtir) = £{w) 
for some i < k, then we must have Ws < 0. Moreover, by (BJ Lemma 3], the descent 
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set of vtir is contained in the descent set of v. We deduce that vUr is also increasing 
up to k. 

FinaUy, it is shown in fS, Thm. 4] that the recursion defining T''{w) terminates 
after a finite number of steps, and hence T^{w) is a finite tree. Moreover, we deduce 
from loc. cit. that if w G Wm then all of the nodes of T''{w) lie in Wn+r- O 

Definition 2. If w £ W^o is increasing up to k and u is a leaf of T^{w), the shape 
of V is the fc-strict partition A associated to v. For any fc-strict partition A, the 
mixed Stanley coefficient e™ is equal to the number of leaves of T^iw) of shape A. 

The next result is a type C analogue of equation Q. 

Theorem 1. If w £ Woo is increasing up to k, then we have an expansion 

(17) ^^=4^^^= V e^e. 



E 

A: |A|=^(to) 



where the sum is over k-strict partitions A. 

Proof. The equality (C^^^ = Jw^ is proved in p^ . We deduce from and the 
definition of T^{w) that if w is not fc-Grassmannian, then 

T(fc) _ j{k) 

(k) 

On the other hand, for any fc-Grassmannian element w = w\, we have Jwx — ©Aj 



by p^ . This completes the proof of the theorem. 

Example 2. The 1-transition tree oi w = 31254 looks as follows. 



□ 



3 12 5 4 



3 14 2 5 



4 12 3 5 



3 2 14 5 



3 2 14 5 




1 3 2 4 5 



3 2 14 5 



By Theorem [T] we therefore obtain 

4^4 = ^(2,1.1) +2 6(3,1) +64. 

When fc = 0, Theorem [T] states that for any w £ Woo, 
(18) F^{X)^ J2 e^^?A(X) 

A : \\\=i{w) 
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summed over strict partitions A. A transition based formula for the constants 
in equation (fT8|) was proved by Billey [B]. There are several alternative combina- 
torial descriptions of these numbers in this case, which include a formula in terms 
of Kraskiewicz tableaux |Krl IL2j of shape A. It would be interesting to have an 
analogous tableau formula for the in the general case where A: > 0. Another 
natural question is whether the mixed Stanley coefficients can be used to obtain 
a Littlewood-Richardson type rule for theta polynomials; some positive results in 
this direction are explained in W2A[ 

Example 3. Consider the double mixed Stanley function 

MX:Y/Z)^ J2 G^-i{-Z)Fu{X)G,{Y) 

summed over all reduced factorizations wuv = w with t37,w G Soo- Fix integers 
3, k >0, and suppose that w G Woo is increasing up to k and and is increasing 
up to j. Then cL"^'''' (AT ; F, Z) = ji~^''"\x ;Y/Z). However, the analogue of 
Theorem [T] fails, at least for fixed k. For an example with j = k = 1, one checks 
that £231'^^ cannot be written as an integral linear combination of ^213 '^"^ ^^'^ 
^312 '^^ while 213 and 312 are the only 1-Grassmannian elements of length two in 

Woo- 

2.3. Reduced words and fc-bitableaux. The type A Stanley symmetric func- 
tions were used in jSta] and to express the number of reduced words of a 
permutation w in terms of the numbers of standard tableaux of shape A, for the 
partitions A which appear in equation Q . Similarly, the type C Stanley symmetric 
functions F^, can be used to compute the number of reduced words for an element 
w e Woo , as shown in (HJ IKr| . We proceed to give an analogue of these results for 
the mixed Stanley functions Jw 

We first recall some definitions from fT21 §5] . Let P denote the ordered alphabet 
{1' < 2' < • • • < A;' < 1 < 2 < • • • }. The symbols 1', . . . , fc' are called marked, while 
the rest are unmarked. Let A be a /c-strict partition. A k-bitableau U of shape A is 
a filling of the boxes in A with elements of P which is weakly increasing along each 
row and down each column, such that (i) the marked entries are strictly increasing 
along rows and weakly increasing down columns, and (ii) the unmarked entries 
form a fc-tableau T. Each A:-bitableau U has an associated multiplicity r{U); see 
loc. cit. for definitions and more details. Let {xy)^ = YliX^'YljVj^ where m, 
(respectively rij) denotes the number of times that i (respectively j') appears in U. 
According to [T2[ Thm. 5], we have 

(19) eA(A;y) = ^2'-(^)(xy)^ 

u 

summed over all fc-bitableaux U of shape A. 

If a /c-bitableau U contains exactly m marked entries, we say that U is of type 
m. U is called standard if the entries 1', . ■ • i w', 1, . . . , rt each appear once in U for 
some n; in this case we have r{U) — n. 

Definition 3. Let m be a nonnegative integer with m < k. For any fc-strict 
partition A, define to be the number of standard fc-bitableaux of shape A and 
type TO. We say that a reduced word for w G Woo has type to if the last to letters 
of the word are positive. 
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Following [T2l Thm. 6 and Ex. 9], for any two /c-strict partitions A, fi with /i C A 
and fii < k, the number g^/^ of standard fc-tableaux of shape A//Lt is equal to the 
number of reduced words for w\w'^^ . Furthermore, using e.g. [T2[ Prop. 5], one can 
construct a bijection between the set of reduced words of type m for wx and the 
set of fc-bitableaux of shape A and type to, for any fc-strict partition A. It follows 
that for any m < fc, we have 

hi = E 

summed over all partitions /i C A with |^| = to. The fc = case of the next result 
is due to Haiman [H] and Kraskiewicz [Kr| . 

Proposition 1. Let w G Woo be increasing up to k and let m be an integer with 
< TO < min(fc, ^(w)). Then the number of reduced words of type to for w is equal 
to e™/i^, where the sum is over all k-strict partitions A. 

Proof. It is clear that the number of reduced words of type to for w equals 2~" 
times the coefficient of xi • • ■ XnVi ■ ■ ■ y-m in Jwi where n — £{w) — to. On the other 
hand, this coefficient is also equal to 2" e'^h^, by ITTl) and □ 

It would be interesting to find a bijective proof of Proposition [TJ 

2.4. Multiplication rules. We show here how Theorem [T] and Lemma [2] lead to 
Littlewood- Richardson type rules for the product of two theta polynomials. In the 
case of fc = 0, we observe that the transition equations in [B] give a combinatorial 
rule for the structure constants in the product of any two Schur Q-functions. This 
answers a question of Manivel in the affirmative (compare with }BL1 §4]). 

We will actually work with the Schur P-functions, which are defined by the 
equation Px — 2^^^^^Qx, for any strict partition A. Given two strict partitions fi 
and i^, there are nonnegative integers f^^^ such that 

Pf.p. = Y.f^,Px. 

A 

The agree with the Schubert structure constants on maximal orthogonal Grass- 
mannians 0G(7i, 2n+l), when n is sufficiently large. The same numbers also appear 
in the expansion of skew Schur Q-functions 

(20) Oa/^-E/m-Q- 

in the Q-basis. Combinatorial rules for the /^^ may be found in [Sa | IW] . IStT] IShj . 

The skew Schur Q-functions are known to be equal to certain type C Stanley 
symmetric functions (see |FK21 Thm. 8.2] and |St2i Cor. 6.6]). In fact, we have 
Q\lii = ^wxw^^^ where wx and are the 0-Grassmannian elements associated to 
A and /i (this is a special case of [T2. Thm. 6]). We therefore deduce the following 
result from mi) and ([20| . 

Corollary 1. The coefficient f^^^ is equal to the number of leaves ofT^{wxw~^) of 
shape v, if <Z \, and is equal to zero, otherwise. 

Now let k be any nonnegative integer. The following result is an analogue of 
[BL| Lemma 16] for the functions Jw. 
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Lemma 2. For w S Wn and v e 6*00, we have 

'Jw'Jy — Jwxv- 

Proof. Since ai ■ ■ ■ ag is a reduced word for v e Sm if and only if (ai +n) ■ ■ ■ (ag + n) 
is a reduced word for 1„ x w, we see that Gi^xv — G^, Fi^^xv = F^, and Jy — Ji^xv 
for each n > 1. The reduced words for vu x v are aU obtained by intertwining a 
reduced word for w with a reduced word for 1„ x v. Moreover, given any reduced 
factorization ab = w x v, with b e Soo, we have a = ai x 02 and 6 — 61 x 62 where 
fli , 61 € Wn and 02 , fei , G <5'oo ■ We deduce that 

(ai Xa2)(6i xb2)=wxv 

FaAX)F,^xaAX)GbAY)Gi„xbAY) 

(ai Xa2)(6i Xb2)=wxv 

J2 Fa,{X)G,AY)Fa2{X)G,,{Y) 

a\b\—w, a2b2—v 

= MX;Y)J4X;Y). □ 

We obtain a combinatorial rule for multiplying two theta polynomials, when one 
of the factors is indexed by a 'small' partition (compare with [BLl Cor. 17]). 

Corollary 2. Let fi and v be k-strict partitions with ui < k for all i, and consider 
the product expansion 

X 

summed over k-strict partitions X. Then Lp^^ is equal to the number of leaves of 
T^{w^ X w,y) of shape X. 

Proof. Observe that ui^, G Soo if and only if i/^ < fc for all i. Lemma [5] therefore 
applies and gives the equation 

o o _ j{k) 7(fe) _ jik) 

The result now follows from Theorem [TJ Recall from |BKT21 §5.4] that in this 
situation we have 0,^ — W — det(i?^.+j_i)i j-. □ 

Example 4. For the 1-strict partitions /i — (2, 1) and = 1 we have it;^, x = 
3T254 e W^. Example H therefore gives 0(2,1)01 — 0(2,1,1) + 2 0(3,1) + 04- 

3. Splitting type C Schubert polynomials 

In this section we give splitting theorems for the single and double type C Schu- 
bert polynomials For any k > Q, let Y>fe = {yk+i,yk+2, ■ ■ ■)■ The following 
proposition generalizes the fc = case from |BH1 Thm. 3] . 

Proposition 2. If w E Wca is increasing up to k, then 

(21) <tn,{X-,Y)= J2 J^u\X ■Y)ey{Y^k) 

xv)—'w 

where the sum is over all reduced factorizations x v) = w with v £ 6*00 • The 

Schubert polynomials CwiX ; Y) for w € W^c increasing up to k form a Z-basis for 
the ring r<-''^Y>k] = r<-''^[yk+i,yk+2, ■■■]■ 
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Proof. From the definition ^ we deduce that for any w E Wn, 
(L^{X;Y)= G:n[0,A:]6„[fc + l,n]. 

According to the polynomial 6„[fc + l,n] is non-zero only if w = 1^ x nj for 
some zu G ^oo, in which case &v[k + l,n] ~ ©^(yfc+i, • ■ ■ , J/n)- For all such v, 
we furthermore note that the element u — wv^^ is increasing up to k, and hence 
C„[0, A:] = Ju[0,k] by Theorem [TJ This proves equation ([2T|l . 

Set = ^w{X ;Y) and for each i > 0, let 9i denote the divided difference 
operator from |BH1 §2]. Recall that di€w = ^wsi, if Wi > Wi+i, and = 0, 

otherwise. Since the Qx = for fc-strict partitions A form a Z-basis of T^''\ we 
deduce that dif = for alH < fc and / in the ring F^'^^ [^>fe]- 

Equations (fl7|l and ([2T|) imply that the Cm for w increasing up to k are contained 
in r^'^) [Y>k]. Moreover, the for w G Woo are known to be a Z-basis of T[Y] from 
[BHl Thm. 3]. Given any / G F('=)[Y>fc], we therefore have / = J2w£W^ '^^ ^'^^ 
some G Z. Since 9^/ = for all i < fc, we deduce that only terms £^ with w 
increasing up to fc appear in the sum, completing the proof. □ 

Fix a sequence a : ai < • • • < Op of nonncgative integers and define Yi — 
{Uai-i+i, ■ • ■ ,2/ai} for each i; in particular Yi = if ai = 0. Given any sequence of 
partitions A = (A^, . . . , A^) with A^ ai-strict, we define the nonnegative integer 

(22) ~ ^ ^Ai'-A^ ' ' ' "^AP' 

ui ■ ..Up—w 

where the sum is over reduced factorizations ui ■ ■ ■ Up = w compatible with a such 
that U2, . . . ,Up G Soo, and the integers c^- and e^} are as in (|4]) and (fT7|) . respec- 
tively. 

The number can be described in a more picturesque way as follows. Given a 
permutation vu G 6*00, let T{w) denote the Lascoux-Schiitzenberger transition tree 
associated to vu in |LS3[ §4]. For any reduced factorization ui ■ ■ ■ Up — w compatible 
with a such that U2, . . . ,Up G 5*00, the p-tuple of trees (T°'^(ui),T(u2), . . . ,T{up)) 
is called a grove. The collection of all such groves forms the a-transition forest 
associated to w. The integer is then equal to the number of p-tuples of leaves 
of shape A in the groves of the a-transition forest associated to w. 

Theorem 2. Suppose that w G Woo *s compatible with the sequence a. Then we 
have 

(23) €„(X;r)= J2 Ju^iX ■,Yi)G,,,iY2) ■ ■ ■ G,,^iYp) 

Ul---Up—W 

summed over all reduced factorizations ui ■ ■ ■ Up = w compatible with a such that 
U2, . . . ,Up G Soo- Furthermore, we have 

(24) G:.„(X ;Y) = Y,el e^i {X ; Y,)sx2 {Y2) ■ ■ ■ s^p (Yp) 

A 

summed over all sequences of partitions A = (A^, . . . , A^) with A^ ai-strict. 

Proof. Equation (pS)) follows from (PT|) and the type A Schubert splitting formula 
(fTU)) . Moreover, ^ is obtained from ^ by using equations ©, (©, and dTT]). □ 
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Let A — 1 . . . , A^). An equivalent expression for the coefBcients is 



(25) e|=^e^c^ 



uv—w 



summed over all reduced factorizations uv = w with v G Sqo, where equals 
the number of sequences of tableaux (T2, . . . , Tp) such that col(T2) • • • col(Tp) is a 
reduced word for v, the entries of Ti are strictly greater than a^-i and the shape 
of Ti is conjugate to A* for each i > 2. With this interpretation of the numbers e™, 
the fc = case of formula (I24p is contained in |Yo| §5]. 

Fix a second sequence b : = 61 < • • • < 6g of nonnegative integers and set 
Zj ~ . . .,Zbj} for each j. A reduced factorization ui ■ ■ -Up+q-i = w is 

compatible with a, b iluj{i) = i whenever j < q and i < bq^j or whenever j > q and 
i < o-j-q- Given any sequence of partitions A = (A^, . . . , A^'+''^^) with A"* ai-strict, 
we define 



(nf.\ fw _ til «<,-i ! «p+,_i 

Jii---Up+,_i = 



where the sum is over reduced factorizations u\ - ■ ■ Up+g_i = w compatible with a, 
b such that Ui G Soo for all i ^ q. Using ([7]) and (fTT|) . one can easily obtain an 
equivalent formula for the along the lines of (P5)) . 



Corollary 3. Suppose that w and w ^ are compatible with the sequences a and b, 
respectively. Then (^^{X ;Y, Z) is equal to 



GuAO/Zq) ■ • •G„,_,(0/Z2)J„,(X ; Yi)G„,^,(r2) • • • G„^+,_,(>;) 



summed over all reduced factorizations ui • • • Wp+q-i = w compatible with a, b such 
that Ui € Soo for all i ^ q. Furthermore, we have 

(27) €^{X ■,Y,Z) = Y, fl s^i{QlZq) • • • eA,(X ; Yi) • • • s^,+,-^{Yp) 

X 

summed over all sequences of partitions A = (A^, . . . , AP+'?~^) with A^ ai-strict. 

Proof. The result follows immediately from equations ([7]), pU]) . and Thcorem[21 □ 

According to [IMNl Thm. 8.1], the Schubert polynomials €w{X ; Y, Z) enjoy the 
following symmetry property: 

(28) ^u,{X-Y,Z) = ^^-i{X;-Z,-Y). 

This also follows immediately from equation ([5]). By applying Corollary [3] to the 
right hand side of pS)) . we obtain dual versions of these splitting equations. 



4. SyMPLECTIC DEGENERACY LOCI 

4.1. Isotropic partial flag bundles. Let ^ i? be a vector bundle of rank 2n 
on an algebraic variety B. Assume that E \s a symplectic bundle, i.e. E is equipped 
with an everywhere nondegenerate skew-symmetric form E ^ E C A subbundle 
1^ of -E is isotropic if the form vanishes when restricted to V; the ranks of isotropic 
subbundles of E range from to n. Fix a sequence a : ai < • • • < ap of nonnegative 
integers with Up < n, and set Op+i = n for convenience. We introduce the isotropic 
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partial flag bundle F°(i?) with its projection map p : F'^(E) — > X. The variety 
F''(£') parametrizes partial flags 

(29) E. : O^EoCEiC-'-dEpCE 

with rankEi — n — Up^i^i and Ep isotropic. Here we have identified E with its 
puUback under the map p, and also let (I29p denote the tautological partial flag of 
vector bundles over F''(£'). 

There is a group monomorphism cj) : Wn ^ with image 

(piWn) = {zD e S2n \ + w{2n + 1 - i) = 2n + 1, for all i }. 

The map is determined by setting, for each w = {wi , . . . , w„) £ Wn and 1 < i < n, 

, - f n + 1 - m„+i_i if Wn+i-i is unbarred, 

[ n + Wn+i-i otherwise. 

Let be the set of signed permutations w € Wn whose descent positions are 
listed among the integers ai, . . . , a^. These elements are the minimal length coset 
representatives in Wn/Wa, where Wa denotes the subgroup of Wn generated by the 
simple reflections Si for i ^ {ai, . . . , a^}. 

Fix a flag = Fq C -Fi C • ■ • C C of subbundles of E with rank Fi = i 
for each i and F„ isotropic. We extend any such flag to a complete flag F, in E by 
letting Fn+i — F^_ - for 1 < z < n; we call the completed flag a complete isotropic 
flag. For every w € W^ and complete isotropic flag F, of subbundles of E, we 
deflne the universal Schubert variety C F''(£') as the locus of a g F°{E) such 
that 

(30) dim(£;^(a) n F,(a)) > # { i < r | (j>{w){i) > 2n - s } V r, s. 

The variety Xw is an irreducible subvariety of F^{E) of codimcnsion £{w), and may 
be regarded as a universal degeneracy locus. Formulas for the classes [Xw] in the 
cohomology or Chow ring of F°{E) pull back to identities for corresponding loci 
whenever one has a symplectic vector bundle V and two flags of isotropic subbundles 
of V, following |Fu3| . Moreover, they are equivalent to formulas which represent 
the Schubert classes in the T-equivariant cohomology ring of isotropic partial flag 
varieties, as observed e.g. in [Graj . 

4.2. Full flag bundles and the geometrization map. Consider the full flag 
bundle F(£') = p'"'^'^' - '""^^ (i?) parametrizing complete isotropic flags of subbun- 
dles E, in E. Let X = (xi, . . . ,x„) and Y = (yi, . . . ,yn). According to |Fu2| §3], 
the cohomology (or Chow) ring H*(F(£'),Z) is presented as a quotient 

(31) H*(F(£;)) -H*(B)[X,Y]/J„, 

where J„ denotes the ideal generated by the differences ei(x^, . . . , x^)— ei(y^, . . . , y^J 
for 1 < i < n. The inverse of the isomorphism ((3T|) sends the class of x^ to 
-ci{En+i-i/En-i) and of y^ to -ci(F„+i_i/F„_i) for each i with 1 < z < n. 

The ring H*(F(£')) may be used to study the cohomology of any isotropic par- 
tial flag bundle F'^{E), because the natural surjection F(£') F"(£') induces an 
injective ring homomorphism l : H*(F''(£')) H*(F(i?)). The tautological vector 
bundles Ei, Fj, the universal Schubert varieties, and their cohomology classes on 
F''(£') pull back under t to the homonymous objects over F(£'). 

The Schubert varieties X^ on F{E) are indexed by w in the full Weyl group 
Wn- Furthermore, the type C double Schubert polynomials itw{X;Y, Z) represent 
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their cohomology classes [Xw] in the presentation ()31|) . but only after a certain 
change of variables. Ikeda, Mihalcea, and Naruse |IMN| §10] provide a different 
way to connect these Schubert polynomials to geometry, which we will adapt to 
our current setup. For a closely related but distinct approach in the case of single 
Schubert polynomials, see |Tlj . 

The key tool is the following ring homomorphism from |IMNj , which we call the 
geometrization map: 

7r„:r[y,Z]^H*(B)[X,Y]/J„. 
The homomorphism tt^ is determined by setting 

r 

MqriX j) = J2 e^{X)hr-^{Y) for all r, 

i=0 

T^niUi) = -Xi for I <i <n, 7r„(yi) = for i > n, Tr„{zj) = yj for each j. 

It follows from [IMNI, Prop. 7.7] that for w € Wn, the geometrization map 7r„ maps 
; Y, Z) to a polynomial which represents the universal Schubert class [Xw] in 
the presentation ((3T|) . Furthermore, we have 7r„(£u,) e J„ for w G Woo \ Wn- 

If V and V are two vector bundles with total Chern classes c{V) and c{V'), 
respectively, we denote s\{c{V) — c{V')) by S\{V — V). We similarly denote the 
c\a.ss<dx{c{V) — c{y)) by Qx{V — V). To state our main geometric result, let Qi — 
E/Ep, Q2 = Ep/Ep-i, . . . ,Qp — E2/E1. Consider a sequence b : = bi < ■ ■ ■ < bq 
with bq < n, and set Q2 = Fn/Fn-b2, ■ ■ ■ ,Qq = Fn-b,_J Fn-b^- 

Theorem 3. Suppose that w £ W and that w^^ is compatible with b. Then we 
have 

[Xni] = S(^x^y{Qq) ■ • • S(A5-1)' ('92)6a9 (Ql - Fn)s x<,+ i {Q2) ' ' ' Sap+9- 1 (Qp) 

A 

= *Ai(-F«+6,_i - Fn+bJ- ■ ■ QX^E ~Ep-Fn)--- Sap+,-i(S2 " ^^l), 

A 

where the sum is over all sequences of partitions X — {X^ , . . . , X^^''^^) with X'' 
ai-strict, and the coefficients fx are given by \20j} . 

Proof. The variables for \ < i < n give the Chern roots of the various vector bun- 
dles over F"(i?). In particular the Chern roots of Qi are xi, . . . ,x„, — xi, . . . , — x^j, 
while those of Qj. for r > 2 are — x^^ j^+i, . . . , — x^^. Similarly the Chern roots 
of Fn+i~r are represented by — y^,...,— y„ for each r. When A: = ai we have 
driX -,¥1) = X][=o 1r-i{X)ei{yi, .. .,yai) for any r > 0. Therefore, we obtain 

7r„(i?,(X;yi)) = e,_,_,(X);i,(Y)e,(-xi,...,-x,J 

= ^er_j(xi, . . . ,x„, -xi, . . . , -XqJ/ij(Y) = Cr{Qi - F„) 

and hence 7r„(8A(-'^ ; Yi)) = Qx{Qi — Fn), for any ai-strict partition A. Moreover, 
for any partition fi and ?■ > 2, we have 

7r„(s^(Fr)) = SM(~XQr-l + li • • • I ^XqJ = Sf^{Qr), 

while 

TTniSf_t{0/Zr)) = S^' (-yfc^.^ + l , . . . , -yb^) = Sf^'{Qr) = S^(F„+6^_j - Fn+b^). 



18 



HARRY TAMVAKIS 



We deduce that 7r„ maps formula (j24|l onto the desired equahty. □ 

5. GlAMBELLI FORMULAS FOR SYMPLECTIC FLAG VARIETIES 

5.1. Partial isotropic flag varieties. Equip the vector space E = C^" with a 
nondegenerate skew-symmetric bihnear form. Fix a sequence o : ai < ■ ■ ■ < ap 
of nonnegative integers with ap < n, and set a^+i — n. Let X(a) be the variety 
parametrizing partial flags of subspaces 

Q = Eo(lEiC-- - (lEpdE 

with din\Ei — n — Op+i_i and Ep isotropic. The same notation will be used to 
denote the tautological partial flag E, of vector bundles over X(a). 

A presentation of the cohomology ring of X(a) as a quotient of the symmetric 
algebra on the characters of a maximal torus in Sp2„ is well known (Boj . We will 
give here an alternative presentation using the Chern classes of the tautological 
vector bundles over X(a). Let Qi ~ E/Ep,Q2 — Ep/Ep^i, . . . ,Qp+i = Ei and 
set ai — Ci{Qi) for 1 < i < n + ai and = Cj{Qr) for 2 < r < p + 1 and 
1 < J < ir — flr-i- We then have the following result 

Proposition 3. The cohomology ring H*(X(a),Z) is presented as a quotient of 
the polynomial ring Z[(Ti, . . . , (T„+ai, c^, . . . , c^^-aiJ ■ ■ ■ i "-i^ : ■ • ■ i cj^-op] modulo the 
relations 

<let{al+j-^)l<^^J<r = {-ly ^ 4 ' ' ' 'Vf ' I < r < n + oi 

i2H hip+i=r 

and 

n+ai ~r 

(32) + 2 ^ {-\yar+^ar-^ = , ai + 1 < r < n. 

i=l 

Proof. Let IG = IG(n — oi, 2n) be the Grassmannian parametrizing isotropic sub- 
spaces of E of dimension n — oi. According to [BKTll Thm. 1.2], the cohomology 
ring of IG is isomorphic to the polynomial ring generated by the Chern classes 
= Ci{Qi) and the Chern classes of Ep, modulo the relations 

det(fTi+j_j)i<ij<r = {-lYcr{Ep) , I <r <n + ai 

coming from the Whitney sum formula c{Ep)c{Qi) — 1, as well as the relations ([5^ . 
The map X(a) — ^ IG sending E, to Ep realizes X(a) as a fiber bundle over IG with 
fiber a partial SL„_aj fiag variety. We deduce using e.g. Gr, Thm. 1] that H*(X(a)) 
is isomorphic to the polynomial ring H*(lG)[c^, . . . , c^^-ai ' ■ ■ • > ^i^^ ^ ■ ■ ■ j c^-op] mod- 
ulo the relations 

12 H hip+i=r 

The proposition follows by combining these two facts. □ 

5.2. Giambelli formulas. Our choice of the special Schubert classes on isotropic 
Grassmannians agrees with the conventions in [BKTlj . Let F, be a fixed complete 
isotropic flag of subspaces in C^". For each w G W°-, the Schubert variety Xw{F,) 
in X(a) is defined by the same equation ([50)) as before. Let E'^ = E/Ep+i^j for 
1 < i < The Chern classes Ci{E'j) for all i,j are the Schubert classes on X(a) 
which are puUbacks of special Schubert classes on symplectic Grassmannians. By 
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definition, they are the special Schubert classes on X(a), and they generate the 
cohomology ring H*(X(a)) by Proposition [3l Speciahzing Theorem [3] to the case 
when the base _B is a point gives the next result. 

Corollary 4. For every w € , we have 

[^-] ^ E {Qi)sx- (Q2) • ■ • SAP {Qp) 

A 

A 

in H*(X(a)), where the sums are over all sequences of partitions A = (A^, . . . , A'') 
with A^ ai-strict, and the coefficients e™ are given by \22^) . 

6. Splitting orthogonal Schubert polynomials 

In this final section we discuss the form of our splitting results for the orthogonal 
groups; we will work throughout with coefficients in the ring Z[i]. For w e VFq o, let 
s{w) denote the number of i such that w{i) < 0. It follows e.g. from [BHillMN] that 
the polynomials S„ = 2^'^^^^€w represent the Schubert classes in the (equivariant) 
cohomology ring of odd orthogonal flag varieties. Therefore the solutions to the 
Schubert polynomial splitting and Giambelli problems for types B and C are es- 
sentially the same. We will describe the splitting theorems for the even orthogonal 
groups below; the story is entirely analogous to the symplectic case. 

The elements of the Weyl group Wn for the root system D„ may be represented 
by signed permutations, as in e.g. [B[ IKTl] . The group Wn is an extension of Sn 
by an element sq which acts on the right by 

(ui,M2, . . . ,U„)so = (U2,U1,U3, . . . ,W„). 

Let Woo = U„W„ and N = {0, 1,2,.. .}. A reduced word of w G Woa is a sequence 
ai ■ ■ ■ Oi of elements in N such that w = Sai ■ ■ • Sag and £ = £{w). We say that w 
has descent at position r > if £{wsr) < £{w), where Sr is the simple reflection 
indexed by r. If k > 2, we say that an element w G Woo is increasing up to k if it 
has no descents less than k; this means that |wi| < W2 < ■ ■ ■ < Wk- We agree that 
all elements of Woo are increasing up to and increasing up to 1. 

For A; e N \ {!}, an element w S Woo is fc-Grassmannian if £{wsi) — £{w) + 1 
for all i ^ k. We say that w is 1-Grassmannian if l{wsi) = £{w) + 1 for all i >2. 
Following [BKTli §4], to any fc-strict partition A we associate a number in {0, 1, 2} 
called the type of A, and denoted type{A). Here 'type' is a multi- valued function 
such that type(A) = if A has no part equal to fc, and type(A) G {1,2}, otherwise. 
The geometric significance of the type of A is explained in |BKT1[ §4.5]. 

Given a /c-Grassmannian element w G Wn, there exist unique strict partitions 
u, V of lengths fc, r, and n — k ~ r, respectively, so that 

W = {Uk, ■ ■ • ,Ul,Cl7 • ■ ■ Xr-,'"n-k-r-, ■■■,Vi) 

where Uk is equal to Uk or 11^, according to the parity of r. If /z; = + i — — 1 + 
I > ^ corresponds to a typed fc-strict partition A such that the 

lengths of the first k columns of A are given by /ii, . . . , /ifc. The part of A in columns 
fc + 1 and higher is given by (Ci — 1, . . . , Cr — 1); here it is possible that = 1, so 
that the sequence ends with a zero. Finally, if type(A) > 0, then Uk is unbarred 
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if and only if type(A) = 1. This defines a bijection between the fc-Grassmannian 
elements of Woo a-nd the fc-strict partitions of all three possible types; we let w\ 
denote the element of Woo associated to the typed fc-strict partition A. 

Following |Llj, we will use the nilCoxeter algebra W„ of Wn to define type D 
Stanley symmetric functions. Wn is the free associative algebra with unity gener- 
ated by the elements uo, ui, . . . , Un-i modulo the relations 

= i>0 ; 

UqUi = UiUq 

uoU2Ua = U2U0U2 

UiUj — UjUi j > i + 1, and 7^ (0i2). 

For any w G Wn, choose a reduced word ai • • ■ ai for w and define — Ua^^ . . . Ua^- 
We denote the coefficient of Uw G Wn in the expansion of the element / G Wn by 
(fjw). Let t be an indeterminate and, following [Lll 4.4], define 

D{t) = (1 + tUn-l) • • • (1 + iM2)(l + tUi){l + <Uo)(l + tU2) ■■■{!+ tUn-l)- 

According to |L1[ Lemma 4.24], we have D{s)D{t) = D(t)D{s) for any commuting 
variables s, t. If D{X) — D{xi)D{x2) ■ ■ ■ , then the functions Ew{X) defined by 

E^{X) = {D{X),w} 

are the type D Stanley symmetric functions, in agreement with |BH| §3]. 
Next, define 

D^{X ■,Y,Z) = ^i„_i(zn-i) ■ • • Ai{zi)D{X)Ai{yi) ■ ■ ■ An-i(yn-i), w'j . 

The polynomials Dw{X ; Y) :— T>w{X ; Y, 0) are the type D Billey-Haiman Schubert 
polynomials, and the Dw{X ;Y, Z) are their double versions studied in jlMNj . If 
w = wx is fc-Grassmannian, then ^^^{X ]Y) is an eta polynomial Hx{X lY); the 
H\ are defined using raising operator expansions analogous to ^ in f BKT4| . When 
fc = 0, we have that A is a strict partition and H\{X ; Y) = E^^ {X) = P\{X) is a 
Schur P-function. 

Definition 4. Given w G Woo, the type D mixed Stanley Junction Iw{X;Y) is 
defined by the equation 

I^{X-Y)^ {D{X)A{Y),w) = J2 Eu{X)G4Y) 

uv—w 

summed over all reduced factorizations uv = w with v G 6*00. 

One checks that if w is increasing up to fc, then /if — Dw"^ {X ;Y); in particular, 
if w = w\ is fc-Grassmannian, then 1^} = H\. Furthermore, it follows from [B| 
Thms. 4, 5] that the Iw satisfy the type D transition equations 

l<i<r i^r 
C(ratrati^)=i!(ra) « (™ t^s * ) = « (™ ) 

where r is the last positive descent of w and s is maximal such that Wg < Wr- 

For any w G Woo which is increasing up to fc, we construct the fc-transition tree 
T'^(w) with nodes given by elements of Woo and root w as in ^'2.'2^ Let r be the 
last descent of w. If w = 1, or fc ^ 1 and r = fc, or fc = 1 and r G {0, 1}, then set 
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T'^{w) — {w}. Otherwise, let s = inax(z > r \ wi < Wr) and = $i(w)U$2(?«), 
where 

$l(w) — {wtrstir I 1 <i <r and (.{wtrstir) — ^(w)}, 
$2(^1) = {wtrstir \ i ^ r and £{wtr.stir) = £{w)}. 

To define T^{w)^ we join it; by an edge to each v G and attach to each 

V G its tree T^{v). 

The assertions of Lemma [T] remain true for T''{w), with similar proof. When 
k — 0, this is contained in (Bj Thm. 4]. For the case when r > k > 1, = 0, 

and Ws > 0, one observes that wtrstir G $2(w)- We deduce that for any w £ Woo 
which is increasing up to k, 

(33) dW^/W^ ^ rf-^^ 

A : \\\=i(w) 

where the sum is over typed fc-strict partitions A and denotes the number of 
leaves of T'^{w) of shape A. Moreover, for any such w, we have 

(34) ^UX;Y)^ I^u\X-Y)e,{Y>k) 

where the sum is over all reduced factorizations u(lfe y. v) = w with v G Soo- 

Using equations ([55]) and (|34p . we obtain splitting theorems for the single and 
double type D Schubert polynomials J)^,, as in iJ31 Fix two sequences a : ai < • ■ • < 
Op and b : = 61 < • • ■ < 6g of nonnegative integers and set Yi = {yai^i+i, ■ • • , IJai } 
and Zj . . . , } for each i, j. 

Theorem 4. Suppose that w G Woo is compatible with the sequence a. Then we 
have 

2)^(X;r)= IuAX;Yi)GuAY2)---Gu,{Yj>) 

U\ ■ --Up—W 

summed over all reduced factorizations ui ■ ■ ■ Up — w compatible with a such that 

U2, ■ ■ ■ ,Up e Soo- 

Given any sequence of partitions A = (A-*^, . . . , Ap+'~^) with A' ai-strict and 
typed, we define 

W _ Ui ,11, tlg+1 Up + q-1 

UX — / , "-Ai ' ' ' ''A9-1 A9''A9+i ' ' AP+<J-i ' 

where the sum is over reduced factorizations ui ■ ■ ■ Up+^-i — w compatible with a, 
b such that m G 5*00 for all i ^ q, and the integers c^- and d^l are as in ([4]) and 
respectively. 

Corollary 5. Suppose that w and are compatible with the sequences a and b, 
respectively. Then ^w{X ; Y, Z) is equal to 

Y Gu, (O/Z,) ■ ■ ■ Gu,_, {0/Z2)Iu, {X ; ri)G„,+, {Y^) ■ ■ ■ G„^+,_, (Ip) 

«i---Up+,_i=io 
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Table 1. Schubert polynomials with w increasing up to 1 



w 
















1 
i 






1 
1 


213 


I 


^1 


213 


I 


^1 


132 


2 




2T3 


Q 


^1 


2T3 


ni 




132 


2 


^ H' 4- 1/^ 
1 ^ -0 1 y2 


312 


21 


'-'(1,1) 


T23 


ni 


rL2 


231 


12 


+ {-)i llo 


312 


21 


H(i.i) 


123 


101 




312 


20 


H', ^ 
-"(1,1) 


3T2 


021 


0(2,1) 


231 


12 


^-^ 2 -'^ i y A 


321 


212 


(""I / n -f \ —1— ("I / I 1 \ 1 


23T 


02 


Ho A- H'. yo 


23T 


012 


("^•j -1- (~)o ?/o 


T32 


201 


3 


132 


2101 


04 


321 


021 


^(2,1) 


321 


1021 


Bi"^ 1 "1 


321 


120 


^(2,1) 


132 


1012 


04 + 03 2/2 


T32 


012 


H3 + H2 2/2 


32T 


0212 


04 + 0(3,1) + 0(2,1) y2 


321 


212 


H3 + H(2,i) + -H'{i,i) 2/2 


231 


21021 


0(4,1) 


32T 


202 


H:i + -^(2,1) + -^(1,1) 2/2 


32T 


01021 


0(3,2) 


231 


2021 


-ff(3,l) 


123 


21012 


05 + 04 y2 


231 


2120 


-^(3,1) 


312 


10212 


0(4,1) + 0(3,1) 2/2 


T23 


2012 


Hi + 1/3 y2 


23T 


021021 


0(4,2) 


3T2 


0212 


H{3,1) + -^(2,1) 2/2 


213 


210212 


0(5,1) + 0(4,1) 2/2 


312 


1202 


-f^(3,i) + -^(2,1) y2 


3T2 


010212 


0(4,2) + 0(3,2) 2/2 


132 


12021 


-ff(3,2) 


132 


1021021 


0(4,3) 


2T3 


20212 


-ff(4,l) + -H^(3,l) 2/2 


2T3 


0210212 


0(5,2) + 0(4,2) 2/2 


213 


21202 


-^^(4,1) + -^^(3,1) 2/2 


123 


10210212 


0(5,3) + 0(4,3) y2 


123 


120212 


-ff(4,2) + -H'{3,2) 2/2 



summed over all reduced factorizations ui ■ ■ ■ Up+^-i = w compatible with a, b such 
that Ui G 5*00 for all i =/= q. Furthermore, we have 

(35) iX:Y,Z) = J2 dl ^x^ (0/Z,) • • • i/^, (X ; Fi) • • • s^p+.-i (5^p) 

A 

summed over all sequences of partitions A = (A^, . . . , A^+'^^^) with ai-strict and 
typed. 

In the same manner as for the symplectic groups, the above splitting results 
imply Giambelli and degeneracy locus formulas for the orthogonal groups. We use 
the Giambelli formula for even orthogonal Grassmannians from [BKT4) in psp . 

Table [1] lists the Billey-Haiman Schubert polynomials for the root systems of 
type C3 and D3 indexed by the elements w in the respective Weyl groups which 
are increasing up to 1. In each case, the polynomial is written as a positive sum of 
k = 1 thcta and eta polynomials in the variables {X, j/i) times Sj(y2) for j G {0, 1}. 
The primed eta polynomials H'^ are indexed by 1-strict partitions A of type 2. 
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